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I Abstract 
(N ■ 

■ In this paper we show that in a multiclass Markovian network with unit rate 

^ ! servers, the condition that the average load p at every server is less than unity is 

indeed sufficient for the stability or positive recurrence for any work conserving 
CN ■ scheduling policy and class-independent routing. We use a variation of the positive 

, recurrence criterion for multidimensional discrete-time Markov chains over count- 

able state spaces due to Rosberg (JAP, Vol. 17, No. 3, 1980) and a monotonicity 
argument to establish this assertion. 
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1 Introduction and Overview 



^ \ In this paper we show that the condition that the load p = Yli=i Pi <^ where Jm denotes 

the number of classes that visit node m at every work conserving unit rate server of a 
' open multiclass Markovian network is sufficient to ensure the stability of the network 

with class-independent routing. We prove this via a variation of the positive recurrence 
criterion for multidimensional discrete-time Markov chains over countable state spaces 



\ due to Rosberg [1]. 

Much research effort has focussed on studying the stability of multiclass queueing 
networks in the last two decades and the amount of the literature thus is understandably 
^ . vast. Much of the research on this topic was fueled by the example of a queueing network 

in a deterministic setting that Lu and Kumar studied in [2]. This paper showed the 
existence of a scheduling policy under which a certain network becomes unstable even 
though the load on each work conserving server in the network is less than unity, which 
was counter intuitive given the well known fact that the load is less than unity is a 
sufficient and necessary condition for the stability of a single queue with multiple classes 
under any work conserving policy. The appearence of the paper [2] was followed by a 
number of papers [31 HI El |6l [3 13 [9l [TOl [TT] that either established the positive assertion 
that multiclass networks are stable under specific settings and policies, or proved that 
under certain specific scheduling policies for which the usual traffic condition p < 1 at 
every work conserving server is not sufficient to ensure the stability of the network. 

Foss and Rybko [H] proposed a condition, called Jackson-type condition, under which 
they showed that p < 1 at every server is a sufficient condition when a FIFO scheduling 
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policy used under renewal assumptions on arrival and service processes. The Jackson- 
type condition roughly means that the routing mechanism is "almost independent" of 
customer classes. In this paper we also assume that the routing is class independent and 
we show that when arrivals are Poisson and service times are exponential, the network is 
stable under the usual traffic conditions for any work conserving scheduling policy (that 
could differentiate between classes). In the network models considered in [2], [7], and [6], 
it should be noted that the customers follow a predetermined fixed route before exiting 
the network and hence the routing is not class independent. This is different from the 
situation considered by Kelly [12] where it is shown that multiclass Markovian networks 
with class independent routing are stable under FIFO and PS scheduling under the usual 
condition. This paper essentially extends those results to any work-conserving scheduling 
policy. 

2 The Stability Analysis 

A common technique to establish ergodicity of countable space Markov chains is through 
Foster-Lyapunov approach which consists of finding an appropriate Lyapunov function 
that satisfies Foster's criterion [13] . Though this approach has proved to be very success- 
ful for one dimensional Markov chains, finding such a function proved to be very difficult 
in the case of multidimensional Markov chains especially networks. Rosberg [T] extended 
Foster's criterion [13] for J > 2-dimensional Markov chains by requiring existence of 
J Lyapunov functions, one for each coordinate of the process. But the applicability of 
his criterion becomes limited when the state definition for the Markov chain does not 
have a fixed dimension. For example, the well known FCFS scheduling policy requires 
a variable-length dimension for the state definition. Also, depending on the modeling 
assumptions, one may only require a fewer component Lyapunov functions than the di- 
mensionality of the underlying state space. Thus, we propose a variation of Rosberg's 
criterion that expands the scope of the applicability that we use to establish stability for 
the multiclass network model we consider. 

Let A* be a countable set of states over which the irreducible, aperiodic, and discrete- 
time Markov chain {X"-, t?, > 0} takes its values. For any integer A; > 1, define {p'^y, x,y E 
to be the fc-step transition probability law of the Markov chain {X",n > 0}. For any 
subset B C X, we know that the limk^oo Pxb — lii^fe^oo J^yeePxy = '^i^) — exists and 
is independent of the initial state x. For any nonnegative-valued function V on X, let 
us define A'^V^x) = '^yPxy^iv) ~~ V{x) to be the fc-step drift of the function V in state 

X. Let c* > denote the limit limfc_j.oo ^ and is independent of x. The following 
definition was originally proposed in [T]. 

Definition 2.1 The sequence k > l| zs said to be uniformly upper bounded 

(UUB) if, for any 5 > 0, there exists a positive integer K such that ^ ^^^^ < c* + 6 for 
k > K and x G X. 

We now propose a variation of the Rosberg's positive criterion [1]. In the rest of the 
paper, we will use the same notation X to denote both the state space and its subsets. 
The distinction is made through the usage of subscripts, i.e., Xj denotes a subset. 

Theorem 2.1 (A variation of Rosberg's criterion [T]) Let J > 2 be an integer. 
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Assumption 2.1 There exists a collection V = {Vi,V2, . . . ,Vj} of partitions of the set 
X where Vj = and nonnegative-valued functions {Vj{x),x E X} for I < j < J 

such that the drift AVj{x) of the function Vj in the state x has the following form: 



where ej > and r]j > 0. 



Assumption 2.2 There exist partitions {^j,/t,^^^}, k >1 and 1 < j < J , of the set 
X with the following two properties: 

(i) piy = 0, < I < k - 1, for X G A^i, and y G Xj 

(a) rijAj^k is a finite set 

Assumption 2.3 For 1 < j < J, the sequence |— ~ "'"} ^'^ uniformly upper 
bounded (UUB). 

Then the Markov chain > 1} is positive recurrent. ■ 

Before we provide a proof of Theorem 12 .11 we first discuss how our variation is different 
from the original theorem in pQ. 

In [T] it is assumed that the the state space of the MC model denoted hy X = Tj]^, the 
J-dimensional non-negative integer space, for some integer J > 2 and an equal number J 
partitions denoted by [Xj,Xj^, 1 < j < J, of the countable space X, and also the same 
number J of Lyapunov functions {Vj{x),x G X}, 1 < i < J. In the variation proposed 
in this paper, we do not require the countable space X to have a fixed predetermined 
dimension. This corresponds to the Assumption 12.11 of Theorem 12.11 Hence we are free 
to choose an appropriate number of Lyapunov functions and the corresponding suitable 
partitions of the state space X. We believe this generalization is useful in many situations 
of interest when the state space is not of fixed predetermined dimension. 

For instance, when the FIFO scheduling policy is implemented at a server the state 
of each queue is characterized not just by the total number of customers present in the 
queue, but also by the class of the customers and the relative order of arrival of the 
customers of different classes. Thus the state description of the server is not of fixed 
dimension but of variable dimension, and potentially can grow very large depending on 
the number of customers. 

Proof: Since the proof of Theorem 12.11 is almost identical to the Proof of Theorem 
2 in [1] we provide here only the main argument, omitting many intermediate results and 
supporting arguments which can be found in [1]. 

Let us fix an arbitrary d > 0. From Assumption 12.31 of Theorem 12. H it follows that 
there exits a positive integer K such that — < c* + 5 for A; > K and 1 < j < J. Let 
us pick one such K, and then introduce the set of functions \^g^{x)]x G X^ such that the 
following holds: 

A^V,{x) = -gfix) + K{c* + S) (2) 

Two observations on the functions g^ are in order: the first and the obvious observation is 
that gf{x) > for a; G A". Also, since A^Vj{x) < — Ke^ for x G ^ (Lemma 2 of [1]), we 
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have that gf{x) > (c* + 5 + e-,) for x E -^j^k- ^ — niinj ej and 6 = miiij As a result, 
we have the obvious deduction that max^ gf{x) > mmj K (c* + 5 + e^) = (c* + 5 + e) for 
X G Ujv4^ Hence maxj gf{x) < miuj K{c* + 5 + e) imphes that x G HjAj^K- We should 
note that x G njv4j^fc neec? not imply that maxj ^'^^(a:) < K (c* + 5 + e). 

Denote by E^; ((^^^(X")) the expectation of gj"{X"') given that = x and by Px{X" G 
A) the probability that G A given that X^ = X. Now 



77, 



where (a) follows from 



Since lim 



n-5>oo „K 



(a) 



yeX 1=0 

E^E!'"[-f'M + '<(c-+i)] 

^ n— 1 

-E-E«(^) + K(c* + 5) 

n— 1 

-^EE.fe'<(^"))+K(c- + i) 



/=0 



we have that i ^"Jq' {gf (X'^)) = K5. Now 



1 / K \ 1 ^ E, 

lim inf —p^ max (x) < K (c* + 5 + e) > 1 — lim sup — > / 77 



> 1 - 



X 



K (c* + 5 + e) 
5:iimsupl$:E.(,]<(X-)) 



j=i ^ /=o 



JK6 



K (c* + 5 + e) 
(c* + 5 + e) ' 



where (b) follows from Lemma 1 in [T]. 

We note that there exists a 5o > such that 1 — , , // , ^ > 0. Define the set 

Ao = jx G A" : max gf{x) < K (c* + 5o + e) | 

We can observe that ^0 ^ '"Ij-^i.fc is a finite set. Hence it follows that for the finite set 
A, 



^ rt— 1 

lim inf - y (X^^ G A) > 



1=0 



Since the chain is assumed to be irreducible and aperiodic, it follows that the Markov 
chain is positive recurrent. ■ 
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3 Multiclass Network Model 



Consider an open queueing network consisting of J work conserving single-servers, each 
with an infinite capacity queue. Let a = 1, 2, . . . denote the class of an arriving customer 
and customers of class a arrive at server j, j = 1, 2, . . . , J as a Poisson process with rate 
\a,j and a customer of class a at node j requires a service time that is exponentially 
distributed with rate ^a,j- After completing service a customer of class a is routed to 
server k with probability Vj^k that is the same for all classes. It is assumed that the 
scheduling policy at any server is a stationary, work- conserving policy. 

One way of constructing this network is to consider a model in which customers from 
outside to the network arrive in a Poisson process of rate A. On arrival, the customer 
joins the queue attached to server j as a class a customer with the probability qa,j such 
that Yliajloi,] ~ ^- Thus \a,j = \qa,j- Once assigned, a customer will retain its class 
till it exits the network. Let R = [r^ denote the routing probability matrix and by 
construction is invertible since by assumption the network is open. 

Let {Aa,j}, where Aaj is the equilibrium rate at which customers of the class a arrive 
to the jth server, be the solution of the following set of equations 

J 

^a,j = Ma,j + Aq, rfc J (3) 
k=l 

written for each class a and for each server j. 

Let us define the load on server j from customers of the class a, then 



Pa, 



Aqj 



p^ 



Let denote the expected number of times a customer of the class a, who is presently 
in the queue j, visits the server k before it departs the network. Then F^^ satisfies the 
following equations: 



J 



J2 -^^-.^r^a.fc = J2^^ 



=1 



(4) 



where 5j^k is the Kronecker Delta function. Let X denote the countable state space for 
the Markov chain {Xn,n > 1}, modeling the evolution of the numbers of customers of 
different classes in various queues. We assume that the state definition is detailed enough 
that in every state x we can obtain the class and positional information of every customer 
in every queue. Let {px.y, x,y ^ X} denote the transition probabilities of the Markov 
chain. 

In the following we show that for a given multiclass network one can find a multiclass 
network but with single customer service rate (i.e., fia,j = such that stability of the 
single customer service rate network implies stability of the actual multiclass network. 
We will see that this reduction of multi service rates to single service rate coupled with 
the class-independent routing assumption helps us easily establish some monotonicity 
properties (Lemma 13. ip in the single service rate network. 

Let S represent a network in which the Poisson arrival rate A and the service rates 
{Pa,]} are such that XlaPaj < ^ fo'^ ea-oh. server j. We now construct a network S' 
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which will be identical to the actual network S except for the Poisson arrival rate and 
average service times of the customers. Specifically, we will choose a A' and /i such that 

A' 

Pa,j < < Pa,j + Va,j < 1 and Y,a Pa,j = ^ Z^a ^a,j < ^ ^'^^^ ^'^^ classcs a and all 
servers j where arbitrarily small positive numbers. We note here that A^^- solve 

the traffic equation ([3]) with A' in place of A. Equivalently 

A' 

max — < /i < min /Xq, , and /i > max > A^ • 

"'i Pa,j + Va,j ' i ^ 

should hold. This becomes feasible when A' is chosen sufficiently small. Thus, we have 
constructed a network S' in which customers arrive at a lower rate and spend longer 
times on an average in each of the servers they visit. Furthermore, the load due to class 
OL at the server j in S' is larger than the corresponding in the actual network S. We 
remark here that even though customers in the network S' have the same service rate \i 
they are still identifiable by their class. 

Following the uniformization technique [H], we will now obtain the discrete-time 
Markov chain queueing models > 0} and {l^,ri > 0} of the multiclass networks 

S and 5', respectively. For the purposes of uniformization, let us choose two numbers 
Q\ and Q2 such that Qi > A + J max^ ,,• /i^ j and > A' + J [i and ^ = ^ where Q\ 
and Q2 are the respective Poisson rates that achieve uniformization for the networks S 
and iS', respectively. Note that Q\ and Q2 are so chosen that at every time epoch the 
probability that a customer arrives is same in both the chains. 

With this construction we now have two {^n} and {YnS that are the same except for 
the probability for service completion at a given time epoch: all servers in the network 
S' are slower than the corresponding servers in the network S. 

Next we demonstrate a construction in which the Markov chains > 0} and 

n > 0} will be defined on a common probability space and then we show that the 
total number of customers in the network S' will always be at least as large as the 
total number of customers in the network S under the condition that both networks are 
initialized in the same state. Let {v4„,n > 1} be a sequence of i.i.d. random variables 
with the distribution p{An = 1) = 7^ = ^ = 1— = 0) where An = I denotes 

an arrival at the nth epoch in both the networks. For 1 < J < J, let {Bj^kik > 1} 
be a sequence of i.i.d. random variables with the distribution p{Bj^k = I) = where 
Bjk denotes the server to be chosen by the /cth departure from the jth server in both 
the networks. Also, by suitable construction we can generate departure events so that a 
when the jth server is non-empty in S as well as S' then a departure from the jth server 
in S' implies a departure from the jth server in S. An implication of this construction 
is that the departure epoch of A;th customer from the jth server in S' will be no sooner 
than the corresponding departure in the network S. Thus the total number of customers 
in the network S' will always be at least as large as the total number of customers in the 
network S. Thus stability of the network S' implies stability of the actual network S. 

Following the arguments above, from now on we concentrate exclusively on the net- 
work with a single customer service rate. 

Theorem 3.1 The condition J2aPa,j < 1 /c"^ 1 < j < J is a sufficient condition for 
positive recurrence of the Markov chain {Xn,n > 1}. ■ 

Before we prove Theorem 13.11 we need to establish the following Lemma. 



6 



Lemma 3.1 Let x E X be a non-zero state and Xj — Ylyex Px y Then Xj — PoXj 
under class-independent routing. u 

Proof: Let the notation X^, t?, > 1, represent the Markov chain whose state at time 
T2, = is X. Let y^{x) = Xla^oj denote the total number of customers in the jth queue 
after n transitions when the state at n = is x. Let now a: be a non-zero state and 0, the 
zero state. We need to show that is stochastically larger than |/"(0); in notation 

>s.t. y^{^)- But this imphes that p^^Xj — Po,Xj n > 1. Following Strassen's 
theorem [15] it is enough to show the same on a common probability space formed by 
the customer arrival process, random variables modeling service times of customers of 
different classes at different servers, and also the random variables that model routing 
decisions of customers at each of the J servers. 

Because of the assumption that customers routing decisions are independent of their 
class, it is not hard to see that >s.t. 2/^(0) and hence p^^Xj — Po,Xj^ n > 1 given 

that a; is a non-zero state. That is, the total number of customers in each queue in a 
non-zero state always dominate the corresponding quantities in a smaller state. ■ 

Proof of Theorem I3.lt By letting Xq, ^ denote the number of customers of the class 
a at the server k, we consider the Lyapunov functions V^(x), one for each server j, that 
were originally proposed in [1]. 



a,k 

k=l 



The Lyapunov function Vj{x) can be interpreted as the virtual work load on the server 
j in state x, whereas Xj = ^^a^aj-, the total number of customers who are presently 
in the queue j, denotes the physical work load on the server j. We now define the sets 
Xj = {x E X : Xj = 0} and then Xj = {x E X : Xj > 1} so that {Xj, Xj^ is a partition 
of the state space X. Define Q = X + Jfi. Our next task is establish the drift 

^(EaA.,,-/^) ifxGA;'= 
h Kj if 2: e Xj 



AV,{x) 



We identify the following events in the system that result in a state transition: 
El An exogenous customer arrival of class a to the server j with probability 

E2 A departure from the jth server that moves to the kth server with probability 



E3 A departure from the jth server that exits the network with probability ^q"^''^'*'^ 
and, 

E4 A transition that takes the chain back to the same state with probability 1 — ^ — 

Z^j-.Xjj^O Q- 

Then we can derive the drift Ay^ ( 



QAVjix) 



'^m,k\ 



a k=l 



k=l 



El 



E2 



E3 



{*) 



E^aj -/Ul{Xj > 0} 
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where (*) follows from the equations (jlj). 

To verify the UUB condition (Assumption 12.31 of Theorem 12.11) . we first note that 

Avr(x) ^ , 1 ^ . ^ . 1 ^ fc 



Xj k=0 y£X^ k=0 

\ ^ n-1 / \ ^ n-1 

E - E pU + E - ^0 - E p',^^ 



n ^ — ' ' \ ^ — ' / n 

k=0 \ a / k=0 

n-1 



^ ^ Px,X'' 

fc=0 



From Lemma 13.11 this implies that ^AV-^{x) < ^A\^"(0), thus proving the UUB 
condition. ■ 
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